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Abstract 

We  are  concerned  with  the  allocation  of  the  base  station  transmitter  time  in  time 
varying  mobile  communications  with  many  users  who  are  transmitting  data.  Time  is 
divided  into  small  scheduling  intervals,  and  the  channel  rates  for  the  various  users  are 
available  at  the  start  of  the  intervals.  Since  the  rates  vary  randomly,  in  selecting  the 
current  user  there  is  a  conflict  between  full  use  (by  selecting  the  user  with  the  highest 
current  rate)  and  fairness  (which  entails  consideration  for  users  with  poor  throughput 
to  date).  The  Proportional  Fair  Scheduler  (PFS)  of  the  Qualcomm  High  Data  Rate 
(HDR)  system  and  related  algorithms  are  designed  to  deal  with  such  conflicts.  The 
aim  here  is  to  put  such  algorithms  on  a  sure  mathematical  footing  and  analyze  their 
behavior.  The  available  analysis  [6],  while  obtaining  interesting  information,  does  not 
address  the  actual  convergence  for  arbitrarily  many  users  under  general  conditions. 

Such  algorithms  are  of  the  stochastic  approximation  type  and  results  of  stochastic 
approximation  are  used  to  analyze  the  long  term  properties.  It  is  shown  that  the 
limiting  behavior  of  the  sample  paths  of  the  throughputs  converges  to  the  solution 
of  an  intuitively  reasonable  ordinary  differential  equation,  which  is  akin  to  a  mean 
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flow.  We  show  that  the  ODE  has  a  unique  equilibrium  and  that  it  is  characterized 
as  optimizing  a  concave  utility  function,  which  shows  that  PFS  is  not  ad-hoc,  but 
actually  corresponds  to  a  reasonable  maximization  problem.  These  results  may  be 
used  to  analyze  the  performance  of  PFS.  The  results  depend  on  the  fact  that  the 
mean  ODE  has  a  special  form  that  arises  in  problems  with  certain  types  of  competitive 
behavior.  There  is  a  large  set  of  such  algorithms,  each  one  corresponding  to  a  concave 
utility  function.  This  set  allows  a  choice  of  tradeoffs  between  the  current  rate  and 
throughout.  Extensions  to  multiple  antenna  and  frequency  systems  are  given.  Finally, 
the  infinite  backlog  assumption  is  dropped  and  the  data  is  allowed  to  arrive  at  random. 
This  complicates  the  analysis,  but  the  same  results  hold. 


1  Introduction:  The  Basic  Algorithm 

Consider  the  problem  where  there  are  a  fixed  number  (AT)  of  users  competing  to  transmit 
data  from  a  single  base  station  to  N  mobile  destinations,  each  moving  independently  of 
the  others,  and  the  possible  rates  of  transmission  of  the  individual  users  are  randomly  time 
varying.  Time  is  divided  into  small  scheduling  intervals  (called  slots).  Until  further  notice, 
in  each  interval  one  of  the  N  users  is  chosen  to  transmit  to  its  destination.  If  user  i  is  selected 
in  interval  n,  then  it  transmits  7yn  units  of  data,  where  {ryn,  n  <  00}  is  a  bounded  (and 
usually  correlated  in  n)  random  sequence,  which  might  also  be  correlated  among  the  i.  They 
need  only  satisfy  some  mixing-type  condition,  specified  in  Section  2. 

Motivation  for  the  work  comes  from  recent  cellular  systems,  such  as  the  Qualcomm  High 
Data  Rate  system  (the  ISO  856  standard),  which  provide  data  connection  via  a  common 
shared  downlink  onto  which  user  transmissions  are  scheduled.  In  that  system,  access  to  the 
link  is  given  one  user  at  a  time  for  a  time  slot  of  fixed  duration  of  1.67  ms.  The  decision  as 
to  which  user  is  to  be  chosen  for  that  slot  is  made  on  the  basis  of  how  much  data  (i.e,  the 
rate)  each  could  transmit  over  the  interval,  as  well  as  the  past  history.  These  rates  take  into 
account  the  estimated  SNR,  which  is  determined  via  measurements  based  on  a  pilot  signal. 
Since  the  time  between  measurement  and  prediction  is  short,  fairly  accurate  rate  predictions 
can  be  made.  Scheduling  decisions  can  take  into  account  Rayleigh  fading  with  a  frequency 
of  a  few  tens  of  Hertz. 

The  selection  of  the  user  at  any  time  is  based  on  a  balance  between  the  current  possible 
rates  and  “fairness.”  One  cannot  choose  the  user  with  the  highest  rate  at  each  slot,  since 
users  with  lower  SNRs  will  be  starved.  The  question  is  how  to  “fair  share.”  Fair  sharing  will 
lower  the  total  throughput  over  the  maximum  possible,  but  it  will  provide  more  acceptable 
levels  to  users  with  poorer  SNRs.  The  algorithm  proposed  by  Qualcomm  performs  this 
sharing  by  comparing  the  given  rate  for  each  user  with  its  average  throughput  to  date, 
and  selecting  the  one  with  the  maximum  ratio.  This  algorithm  is  known  as  proportional 
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fair  sharing  (PFS)  and  is  related  to  the  fairness  criterion  given  by  Kelly  [7]  in  allocating 
connections  over  multiple  links  on  the  Internet. 

Until  further  notice,  it  is  assumed  that  each  user  has  an  infinite  backlog  of  data,  the 
standard  assumption  in  the  literature  to  date.  See  [3,  10],  where  the  channel  behavior  is 
stationary  and  ergodic.  Such  results  indicate  the  gains  that  can  be  made  by  exploiting 
channel  fluctuations  that  result  from  Rayleigh  fading  etc.  Our  assumptions  will  be  weaker. 
Let  the  end  of  time  slot  (i.e.,  scheduling  interval)  n  be  called  time  n.  At  time  n,  the  possible 
rates  {rjjn+i,i  <  N}  for  the  next  time  slot  are  known.  Let  Ii,n+ 1  be  the  indicator  function 
of  the  event  that  user  i  is  chosen  at  time  n  to  transmit  in  slot  n  +  1.  One  definition  of  the 
throughput  for  user  i  up  to  time  n  is  the  sample  average 

n 

@i,n  ^  ]  T ijljj  / '^i 
1=1 

where  Iij+i  =  1  if  user  i  is  chosen  at  time  l  and  is  zero  otherwise.  With  the  definition 
en  =  l/(n  +  1),  (1.1)  can  be  written  in  the  recursive  form  (which  defines  Yn) 

@i,n+ 1  @i,n  d"  @i,n\  @i,n  d~ 


(1.2) 


(1.1) 


An  alternative  definition  of  throughput  discounts  past  values  of  the  ryn.  For  small  positive 
e,  and  the  discount  factor  1  —  e,  the  discounted  throughput  is  defined  by 

n 

e;,„  =  (1  -  <0X»  +  *£(i-  ‘rVk  (1.3) 

(=1 

This  can  be  written  in  the  recursive  form  (which  defines  Yf) 


Otn+l  -  +  e 


Hjn+lTi,n+l 


-e, 


^,7^ 


=  6e  +  eY6 

wi,n  1 


(1.4) 


where  Zfn+1  is  the  indicator  function  of  the  event  that  user  i  is  chosen  at  time  n.  The 
representations  (1.2)-(1.4)  allow  arbitrary  initial  conditions,  which  might  reflect  some  past 
history  or  bias.  The  recursive  representation  (1.2)  allows  the  use  of  other  values  of  the  {e„}. 
For  example,  they  might  go  to  zero  more  slowly  than  1/n,  giving  a  weighting  between  those 
of  (1.1)  and  (1.3).  Owing  to  the  boundedness  of  the  rvn,  the  solutions  to  (1.2)  and  (1.4)  are 
bounded. 

The  value  of  e  is  chosen  to  balance  the  needs  of  estimating  throughput  (requiring  a  small 
value  of  e)  with  the  ability  to  track  changes  in  the  channel  characteristics  (requiring  a  larger 
value  of  e).  In  general,  e  should  be  chosen  small  enough  so  that  it  provides  an  acceptable 
measure  of  the  throughput.  A  useful  guideline  is  ex  [number  of  time  slots  by  real  time  t] 
=  0(t). 
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The  representations  (1.2)  and  (1.4)  are  of  the  stochastic  approximation  form  (see  the 
comprehensive  reference  [8]),  and  the  results  of  stochastic  approximation  theory  will  be  used 
for  their  analysis.  The  possibility  of  using  more  general  queues  and  the  arbitrariness  of  the 
channel  rate  processes  and  of  the  en  illustrate  the  power  of  the  approach.  As  will  be  seen, 
there  are  also  convenient  adaptations  to  multiple  antenna/frequency  systems.  In  addition, 
the  assignment  algorithm  and  convergence  results  will  be  seen  to  be  typical  of  a  large  family 
of  algorithms,  each  corresponding  to  some  concave  utility  function,  which  in  turn  is  actually 
maximized  by  the  associated  algorithm.  Although  the  methods  and  conclusions  exploit 
current  results  in  stochastic  approximation,  they  are  far  from  obvious. 

The  original  proportional-fair  sharing  algorithm  chose  the  user  that  maximizes  in1 

arg  maxjr  in+i/  9in},  (1.5) 

i<N 

or  with  6\n  replacing  9n  if  (1.4)  is  used.  When  all  of  the  current  components  9i)n,i  <  N,  are 
very  small,  there  is  little  sense  in  (1.5),  since  the  current  throughputs  are  all  essentially  zero 
and  there  is  no  reason  to  distinguish  between  them.  We  modify  the  algorithm  slightly  as 
follows.  Let  di,i  <  N,  be  positive  numbers,  which  can  be  as  small  as  we  wish.  The  chosen 
user  at  time  n  is  that  which  maximizes  in 


argmax{rijri+i/(di  +  9^n)}, 


(1.6) 


or  with  9en  used  if  the  algorithm  is  (1.4).  In  the  event  of  ties,  let  us  randomize  among 
the  possibilities.  We  choose  randomization  for  resolving  conflicts.  But  the  end  results  are 
completely  independent  of  how  the  conflicts  are  resolved. 

Discretized  or  quantized  rates.  Suppose  that  the  variables  ryn  are  the  theoretical  rates 
in  that  there  is  in  principle  a  transmission  scheme  that  could  realize  them,  perhaps  by 
adjusting  the  symbol  interval  and  coding  in  each  scheduling  interval.  In  applications,  it 
might  be  possible  to  transmit  at  only  one  of  a  discrete  set  of  rates.  The  algorithms  and 
results  are  readily  adjusted  to  accommodate  this  need.  Continue  to  make  the  assignments 
using  (1.6),  based  on  the  values  of  the  ryn+i,i  <  N.  But  use  the  true  transmitted  rate, 
called  r^i  in  computing  the  throughput.  Then  (1.4)  is  replaced  by 


Qi,n+ 1  “  @i,n  +  6 


rp 

n+ 1'  1 


0i,n  )  —  Otn  + 


(1.7) 


Much  is  known  about  the  behavior  of  algorithms  such  as  (1.2)  and  (1.4);  see,  for  example, 
[8,  Chapters  8].  Under  broad  conditions,  if  the  tracking  parameter  e  in  (1.4)  is  small  and 

1The  original  algorithm,  as  applied  to  mobile  communications,  is  due  to  D.N.C.  Tse. 
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constant,  or  if  n  is  large  in  (1.2),  then  the  path  converges  to  the  solution  to  a  determin¬ 
istic  ordinary  differential  equation,  which  is  computed  from  the  “mean”  dynamics  of  the 
throughput  process. 

As  we  have  just  remarked,  the  path  9e{-)  will  essentially  “follow”  the  solution  to  the 
ordinary  differential  equation  (ODE).  The  ODE  has  a  unique  equilibrium  point  9.  The 
sample  throughputs  of  (1.2)  will  converge  to  9  for  large  n.  With  (1.4),  the  sample  throughputs 
will  congregate  very  close  to  9  if  e  is  small.  The  existence  of  a  unique  equilibrium  is  of 
significance  for  the  performance  analysis  of  the  proportional  fair  algorithm.  This  is  because 
the  equilibrium  state  which  will  determine  the  throughput  of  each  user  and  hence  delay. 

An  interpretation  of  (1.6);  maximizing  a  utility  function.  Define  the  utility  function 

^(0)  =  £log(di  +  0i).  (1-8) 


An  alternative  view  of  (1.6)  is  that  it  maximizes  U(9n+ 1)  —  U(9n )  to  first  order  in  the  en,  as 
seen  by  the  following  argument.  By  a  first  order  Taylor  expansion, 


U(9n+1)  -  U(9n) 


i 


D,n+l-^i,n+l  9i. 

di  T  9i  n 


+  CKen)' 


(1.9) 


Since  Yhih,n+ 1  =  1,  to  maximize  the  first  order  term  we  must  choose  /,n+i  by  (1.6).  It  will 
be  shown  in  Section  3  that  the  rule  (1.6)  maximizes  lim nU(9n)  over  all  other  admissible 
rules.  See  also  the  comments  below  (3.5)  concerning  the  form  (1.8).  The  function  was  not 
chosen  apriori.  It  is  the  unique  one  associated  with  the  original  form  of  PFS. 

The  outline  of  the  paper  is  as  follows.  Section  2  begins  with  some  definitions.  Then  the 
assumptions  are  stated.  They  are  stated  in  a  general  form,  not  only  so  that  we  can  appeal 
to  general  results  in  stochastic  approximation  to  facilitate  the  development,  but  also  so  that 
the  basic  structure  that  is  required  will  be  clear.  The  overall  framework  is  quite  flexible,  and 
there  are  many  useful  variations.  It  is  seen  that  the  assumptions  are  quite  reasonable.  Then 
a  weak  convergence  theorem  for  the  iterative  algorithms  (1.2)  and  (1.4)  is  presented. 

Theorems  2.2  and  2.3,  which  show  that  proportional  fair  has  a  unique  stable  point  and  op¬ 
timizes  the  limiting  utility  lim  U ( 9n ),  are  new.  These  results  depend  heavily  on  the  structure 
of  proportional  fair  and  its  relationship  to  what  are  known  as  monotone  dynamical  systems, 
as  seen  in  Section  3.  The  existence  of  a  unique  equilibrium  is  of  considerable  significance  for 
the  performance  analysis  of  PFS;  otherwise,  the  usefulness  would  be  questionable.  To  date, 
the  primary  alternative  to  such  analysis  has  been  simulation.  Except  for  short  transfers,  the 
equilibrium  state  determines  user  throughput,  hence  delay.  In  Section  4,  we  give  an  example 
that  illustrates  the  quality  of  the  approach  for  modeling  and  analysis  for  the  special  case  of 
Rayleigh  fading. 
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Section  5  is  devoted  to  extensions  of  the  earlier  results.  The  log  concave  utility  is  not 
essential.  See  Subsection  5.1.  The  results  also  hold  if  there  are  multiple  channels  with  more 
than  one  user  assigned  at  once.  This  allows  applications  such  as  scheduling  from  multiple 
transmit  antennas,  see  Subsection  5.2.  In  Section  6,  we  drop  the  assumption  of  infinitely 
backlogged  users.  Suppose  that  (over  the  time  period  of  interest),  there  are  still  a  fixed 
number  of  users  in  the  system,  but  they  create  data  at  random  rates.  This  data  is  queued 
until  transmitted.  Under  natural  conditions,  there  is  still  a  mean  ODE  which  characterizes 
the  flow  of  the  algorithm.  The  queues  themselves  need  to  be  brought  into  the  analysis,  since 
some  might  be  empty  at  a  scheduling  time.  This  complicates  the  analysis,  but  the  basic 
properties  of  and  use  of  the  mean  ODE  have  not  changed. 

The  convergence  theorems  which  characterize  the  asymptotic  behavior  of  the  algorithms 
are  given  in  Section  3,  where  it  is  also  seen  that  argmax  rule  (1.6)  also  maximizes  the  utility 
function  (1.8)  for  small  e  and  large  n.  The  special  case  of  constant  rates  with  discontinuous 
dynamics  is  also  discussed  briefly.  The  utility  function  (1.8)  plays  no  special  role  in  the 
analysis.  In  Subsection  5.1,  we  see  that  other  strictly  concave  utility  functions  can  be  used 
as  well.  This  allows  a  choice  of  tradeoffs  between  the  current  rate  and  throughput  in  making 
the  assignments.  One  advantage  of  the  flexibility  of  the  approach  is  the  ease  with  which 
more  complex  systems  can  be  treated.  To  illustrate  this,  in  Subsection  5.2.  we  discuss  the 
extension  when  there  are  multiple  antennas  or  channels,  and  they  can  be  assigned  either 
individually  or  in  some  coordinated  way. 

Suppose  that  the  users  have  a  large,  but  not  infinite,  amount  of  data,  and  that  new 
users  can  arrive  from  time  to  time.  Then,  during  periods  in  which  the  number  is  fixed,  the 
throughputs  will  follow  the  path  of  the  mean  ODE.  For  PFS  to  work  well,  one  would  need 
to  choose  an  appropriate  initial  condition  for  any  new  user.  It  would  be  best  if  this  were 
an  estimate  of  the  equilibrium  value  for  that  user  in  the  current  situation.  For  example,  if 
we  start  a  user  with  a  throughput  value  of  zero,  then  it  will  take  slots  even  with  very  poor 
current  rates.  In  Section  6,  we  drop  the  assumption  of  infinitely  backlogged  users.  Suppose 
that  (over  the  time  period  of  interest),  there  are  still  a  fixed  number  of  users  in  the  system, 
but  they  create  data  at  random  rates.  This  data  is  queued  until  transmitted.  Under  natural 
conditions,  there  is  still  a  mean  ODE  which  characterizes  the  flow  of  the  algorithm.  The 
queues  themselves  need  to  be  brought  into  the  analysis,  since  some  might  be  empty  at  a 
scheduling  time.  This  complicates  the  analysis,  but  the  basic  properties  of  and  use  of  the 
mean  ODE  have  not  changed. 
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2  Assumptions  and  Convergence  Theorems 


First,  some  definitions  will  be  given.  Then  we  make  a  few  comments  concerning  what  is 
called  weak  convergence.  The  assumptions  are  then  stated  and  discussed,  after  which  the 
theorems  are  stated. 

Definitions.  Define  the  vectors  9n  =  {9i:n,i  <  N},  9en  =  {0-  n,  i  <  N},  and  Rn  =  {rhn,  i  < 
N}.  The  usual  stochastic  approximation  asymptotic  (or  large  time)  analysis  of  the  algorithms 
(1.2)  and  (1.4)  uses  continuous  time  interpolations.  For  each  n,  define  the  shifted  process 
9n(-)  (with  components  #/(•),  i  <  N )  by  9n( 0)  =  9n  and,  for  l  >  0, 


9n(t ) 


9n+i  for  t  E 


'n-\-l— 1  n-\-l  \ 

eki  yy 


k=n 


k=n 


where  the  empty  sum  is  defined  to  be  zero.  Since  the  interpolated  process  #”(•)  starts  at 
iterate  n,  the  behavior  of  9n(-)  as  n  — >  oo  is  that  of  9n  as  n  — >  oo.  Analogously,  define  the 
interpolated  process  9e( •)  (with  components  9f(-),i  <  N )  by  9e(t)  =  9f  for  t  e  [ne,  ne  +  e). 

Weak  convergence.  We  work  in  a  so-called  weak  convergence  setup,  which  is  the  most 
powerful  approach  for  the  analysis  of  such  algorithms  [8,  Chapter  8].  It  is  concerned  with  the 
characterization  of  the  limits  of  the  processes  9n(-)  for  large  n  and  9ef)  for  large  time.  The 
details  of  the  theory  are  not  necessary  for  the  development  of  the  results.  If  we  say  that  9ef) 
converges  weakly  to  a  process  with  constant  value  9.  it  means  that  for  large  n,  the  paths  of 
9£(-)  are  very  close  to  the  point  9,  with  a  high  probability.  The  algorithm  (1.4)  requires  the 
weak  convergence  view,  since  there  will  not  be  probability  one  convergence.  One  would  not 
usually  want  to  use  (1.2)  with  the  step  size  en  =  l/(n+  1)  since  a  few  bad  values  of  the  noise 
in  the  early  stages  can  mess  up  the  behavior  of  the  sample  path  for  a  long  time  to  come,  and 
such  robustness  considerations  require  that  we  have  a  larger  discounting  of  past  values  than 
en  =  l/(n-|- 1)  provides.  The  weak  convergence  approach  gives  us  much  more  flexibility  than 
a  probability  one  method,  where  that  is  possible.  The  discounted  algorithm  (1.4),  or  any 
form  which  is  set  up  to  allow  tracking  in  the  presence  of  time  varying  parameters,  cannot 
converge  w.p.l,  so  that  a  weak  convergence  analysis  must  always  be  done.  More  detail  and 
discussion  is  in  the  comprehensive  reference  [8,  Chapter  8]. 

Assumptions.  The  assumptions  are  quite  weak.  For  convenience  in  applications,  they  are 
stated  in  a  way  that  allows  two  options.  The  first,  which  is  (A2.1),  is  simpler  to  state  and 
requires  stationarity  of  {/?„}.  The  second,  which  is  (A2.2),  is  implied  by  (A2.1)  and  allows 
more  general  behavior,  such  as  deterministic  cycling,  etc.  The  conditions  and  the  problem 
formulation  are  designed  to  take  maximum  advantage  of  the  results  in  [8,  Chapter  8].  Owing 
to  the  boundedness  of  the  Rn,  the  conditions  of  the  convergence  theorems  [8,  Sections  8.2, 
8.4]  are  implied  by  the  forms  used  below.  Let  En  denote  the  expectation  conditioned  on 
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{Rul  <  n;  0O},  the  data  needed  to  calculate  9n.  The  symbol  ri  is  used  as  the  canonical  value 
of  the  components  ryn,  and  9i  and  93  are  used  as  the  canonical  values  of  0^n  and  6j>n,  resp. 
Let  iR^r  denote  the  set  of  points  x  G  MN  with  all  components  nonnegative.  The  requirements 
of  the  first  stochastic  approximation  result,  Theorem  2.1,  are  minimal.  But,  once  the  mean 
ODE  is  characterized,  we  need  to  know  more  about  the  set  of  possible  limit  points.  The  fact 
that  the  the  limit  point  is  unique  and  globally  asymptotically  stable  is  proved  by  using  some 
results  from  dynamical  systems  theory. 

A2.0.  Let  denote  the  past:  {Ri  :  l  <  n}.  For  each  i,n,£n, 

hitn(9,  £n)  —  EnTi,n+lI{ri,n+-i/(di+6i)>rjtn+i/(dj+6j),j^i} 

is  continuous  in  9  e  M+.  Here  9  is  considered  fixed.  Let  <5  >  0  he  arbitrary.  Then  in  the  set 
{9  :  Oi  >  6,i  <  N},  the  continuity  is  uniform  in  n  and  in  fn. 

A2.1a.  {Rn,  n  <  oo}  is  stationary.  Define  h%  ( • )  by  the  stationary  expectation: 

hi(9)  =  Eril{n/(di+0i)>rj/(dj+6j),j^i}i  i  <  N.  (2-1) 


In  (2.1),  9  is  considered  fixed.  Also, 

\Rn'ri,l+lI{ri!i+1/(di+0i)>rjj+1/(dj+dj),jj£i}  ~  hfiO) 


-1  n+ra— 1 

lim  —  'Y' 

i  n — wyi  ‘ 

l=n 


=  0 


in  the  sense  of  probability.  There  are  small  positive  6  and  6 1  such  that 
P  { r^n/di  >  r}JI/id,  -  6)  +  <5i,  j  ^  i}  >0,  i  <  N. 


(2.2) 


(2.3) 


A2.1b.  /?,  is  defined  on  some  bounded  set  and  has  a  bounded  density. 

Comments  on  assumptions  (A2.0)  and  (A2.1).  The  last  part  of  (A2.1a)  is  innocuous 
and  is  used  to  assure  that  when  a  component  9t  is  very  small  there  is  a  nonzero  chance  that 
user  i  will  be  chosen,  no  matter  what  the  values  of  the  other  components  of  9.  This  is  hardly 
a  restriction.  It  guarantees  that  the  mean  rate  function  hfiO)  defined  in  (2.1)  is  positive  when 
9\  is  small,  The  density  assumption  (A2.1b)  is  satisfied  under  standard  physical  assumptions. 
Indeed  all  the  assumptions  hold  under  Rayleigh  fading  if  the  channels  are  independent.  The 
density  condition  is  used  only  to  show  that  the  limit  point  is  unique.  Condition  (2.2)  is  a 
very  weak  form  of  the  law  of  large  numbers,  due  to  the  use  of  the  conditional  expectation 
En.  If  the  conditional  expectation  of  the  transmitted  rate  at  time  /,  given  the  data  to  time 
n,  is  close  to  its  stationary  expectation  for  large  positive  l  —  n,  then  it  holds.  If  the  channel 
rate  process  is  ergodic,  then  the  condition  holds  even  without  the  conditional  expectation. 


So  the  combination  of  the  effects  of  the  conditional  expectation  and  the  division  by  m  gives 
a  very  weak  condition  indeed.  Condition  (A2.0)  asks  that  slight  changes  in  9n  would  change 
the  conditional  (on  data  to  the  present)  expectation  of  the  next  accepted  rates  only  slightly. 
It  is  [8,  condition  (A2.3),  Chapter  8].  The  condition  can  be  weakened  by  the  approach  taken 
in  Section  7,  but  is  good  enough  for  typical  applications  where  (A2.1b)  holds. 

On  the  smoothness  of  h(-).  By  (A2.1b),  h{-)  is  Lipschitz  continuous.  To  see  this,  consider 
the  two  dimensional  case  and  let  p(-)  denote  the  density  of  Rn.  Let  9  e  and  write 
w  —  (di  +  9i)/(d2  +  92).  Then 

hi{9)  =  J  r1I{ri/r2>w}p(r1,r2)dr1dr2, 

which  is  Lipschitz  continuous  with  respect  to  w,  since  the  area  of  the  region  where  the 
indicator  is  not  zero  is  a  differentiable  function  of  w.  The  derivative  of  ht(-)  will  be  continuous 
if  p(-)  is  bounded  and  continuous. 

A  weaker  set  of  conditions.  The  set  of  assumptions  (A2.2a,b)  is  weaker  than  (A2.1a,b)  in 
that  it  covers  it  and  allows  the  distributions  of  the  Rn  to  vary  with  n.  For  example,  they  can 
cycle  in  a  deterministic  way.  A  weak  ergodicity  property  analogous  to  (2.2)  is  still  required. 
In  (A2.1),  the  condition  (2.3)  was  sufficient  to  assure  that  the  solution  of  (2.7)  would  have  all 
components  positive  after  an  arbitrarily  small  time.  We  need  a  similar  property  when  using 
(A2.2a,b)  in  order  to  avoid  the  possibility  that  some  component  will  be  zero  in  the  limit 
due  to  degeneration  of  the  rates.  Because  of  the  possibility  of  nonstationarity,  a  reasonably 
general  sufficient  condition  cannot  be  expressed  as  simply  as  it  was  in  (2.3).  There  are  many 
sets  of  conditions  which  will  assure  it.  For  example,  consider  the  random  variables 
obtained  by  randomizing  among  {Rn+i,l  <  M}  for  some  integer  M.  If  Rff  satisfies  (2.3) 
uniformly  in  n,  then  the  desired  positivity  will  be  assured.  In  order  to  avoid  lots  of  examples, 
we  simply  write  a  sufficient  condition  (2.6)  for  what  is  required.  Analogously  to  the  situation 
with  the  set  (A2.1),  (A2.2a)  is  used  in  the  basic  stochastic  approximation  result,  Theorem 
2.1,  and  (A2.2b)  is  used  to  show  that  there  is  a  unique  limit  point.  Also,  as  noted  above, 
the  use  of  the  conditional  expectation  in  (2.5)  gives  a  very  weak  form  of  the  law  of  large 
numbers. 

A2.2a.  The  limits  ( stationary  expectation  used ) 

-i  n+m— 1 

hi(0)=  lim  -  E  EhiMb)  (2.4) 

m,n—>oo  jji  z ' 
l=n 

exist  and,  for  each  9, 

i  n+m— 1 

lim  -  E  En\hi,i{e^i)-hi(0)]=O,  i<N,  (2.5) 

m,n— »oo  jji  1  L  ’  '  '  i 
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in  probability.  There  are  small  positive  8,61,  and  an  integer  M: 


inf  P  {r di  >  rjtn+i/(dj  -  6)  +  Suj  ±  i,  for  some  l  <  M}  >  0,  i  <  N. 


(2.6) 


A2.2b.  For  any  integer  M ,  consider  the  random  variables  R„  obtained  by  randomizing 
among  the  {Rn+i,  l  <  M}.  There  is  M  such  that  the  R(f ,  n  <  oo,  have  densities  p^(-)  which 
are  uniformly  (in  n)  bounded. 

Note  on  quantized  rates.  If  the  transmitted  rates  are  quantized  as  in  (1.7),  then  redefine 

hi,n(@i£n)  =  ^nritn-\-lI{ri^n+1/(di+Oi)>rjtn+1/(dj+dj),j^i}j 


and  in  (2.2)  use 


hi(9)  —  Er^  I{ri/(di+9i)>rj/(dj+0j),j^i}, 

{riti+1/{d,i+di)>r^i+1/{dj  +0j) ,j^i}  ■ 


The  limit  process  and  mean  ODE.  The  next  theorem  is  a  standard  result  in  stochastic 
approximation.  It  basically  says  that  the  limit  points  of  the  algorithms  (1.2)  and  (1.4)  are 
contained  in  those  of  the  ODE  (2.7). 

Theorem  2.1.  (This  is  [8,  Theorems  2.2  and  2.3,  Section  8.2].)  Assume  the  algorithm 
(1.2),  (A2.0),  and  either  (A2.1a)  or  (A2.2a).  Then  for  any  initial  condition,  9n(-)  converges 
weakly  to  the  set  of  limit  points  of  the  solution  of  the  ODE 


ei  =  hl(d)-ei,  i  <  n. 


(2.7) 


The  same  conclusion  holds  if  the  en  in  (1.2)  is  replaced  by  a  sequence  en  such  that  en 
0,  En  en  =  oo,  and  where  en  doesn't  vary  too  fast  in  that  for  some  sequence  an  — >  oo 


lim  sup  |en+//en  -  1|  =  0.  (2.8) 

n  0  <l<an 

For  algorithm  (1.4),  the  same  conclusion  holds  for  the  sequence  9e(eqe  +  ■)  for  any  sequence 
of  integers  qe.  In  particular,  if  9$  — >  0(0),  then  9e(-)  converges  to  the  unique  solution  to  (2.7) 
with  initial  condition  0(0).  The  conclusions  hold  if  the  discretized  rates  are  used. 

Comment  on  the  proof.  The  conditions  of  [8,  Theorems  2.2  and  2.3,  Section  8.2]  are 
easily  verified.  The  boundedness  of  Rn  implies  the  uniform  integrability  needed  in  (the 
following  cited  conditions  (conditions  (Al.l)-(A1.9)  are  those  in  the  reference)  (Al.l)  and 
(Al.8).  The  quantity  (3n  in  (Al.5)  is  zero.  In  our  case,  we  can  suppose  that  fn  is  defined  on 
a  compact  sequence  space,  so  (Al.7)  is  trivially  satisfied.  The  averaging  assumption  (Al.9) 
is  assured  by  either  (2.2)  or  (2.5).  The  next  two  theorems  will  be  proved  in  Section  4. 
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Theorem  2.2.  Assume  algorithm  (1.2),  (A2.0),  and  either  (A2.1a,b)  or  (A2.2a,b).  The 
limit  point  (9)  of  (2.7)  is  unique,  irrespective  of  the  initial  condition.  So  the  processes  9n(-) 
and  9€(eqe  +  •)  converge  to  9  as  n  — >  oo  ( resp as  e  — >  0  and  eqe  — >  oo,  or  as  e  ->  0  and 
t  — >  cx)) .  The  conclusions  hold  if  the  discretized  rates  are  used. 

Optimizing  the  utility  function  (1.8).  Up  to  now,  we  have  concentrated  on  the  algorithm 
(1.6).  Theorem  2.2  shows  that  there  is  a  unique  asymptotically  stable  limit  point  9  of  the 
ODE  and  algorithm.  We  have  not  addressed  the  optimality  of  the  algorithm.  We  can  see 
intuitively  from  the  argument  below  (1.8)  that  (1.6)  is,  in  the  limit,  a  “steepest  ascent” 
algorithm  for  a  strictly  concave  utility  function.  The  problem  is  that  the  allowed  directions 
of  ascent  at  each  value  of  9  depend  on  9.  Hence  there  is  no  a  priori  guarantee  of  any  type  of 
maximization.  The  following  theorem  is  one  way  of  quantifying  this  idea. 

Theorem  2.2.  Assume  the  conditions  of  Theorem  2.2.  There  is  no  assignment  policy  which 
yields  a  limit  throughput  9^9  such  that  U{6)  >  U (9) . 

3  The  Limit  Point:  Proof  of  Theorem  2.2 

The  proof  depends  on  a  monotonicity  property  of  the  solution  to  (2.7),  which  is  fundamental 
to  the  analysis  of  a  large  class  of  stochastic  algorithms  which  model  competitive  or  coopera¬ 
tive  behavior.  The  importance  of  these  concepts  was  introduced  in  [5]  and  further  discussion 
is  in  [1,  2].  First,  some  general  facts  from  the  theory  of  dynamical  systems  will  be  stated. 

Definitions.  For  vectors  X,Y  €  IiN ,  we  write  X  >  Y  (resp.,  X  >  Y)  if  Xt  >  Y)  for  all 
%  (resp.,  and,  in  addition  X  ^  Y).  If  X,  >  Yi  for  all  i,  then  we  write  X  X  Y .  Let  H,K 
be  subsets  of  MN .  Then  write  H  >  K  (resp.,  H  X  K )  if  X  >  Y  (resp.,  X  X  Y)  for  all 
X  e  H,  Y  e  K.  Consider  a  dynamical  system  in  MN: 

x  —  f(x),  /(•)  Lipschitz  continuous.  (3.1) 

Where  helpful  for  clarity,  we  might  write  x{t\y)  for  the  solution  to  (3.1)  when  the  initial 
condition  is  y,  and  use  analogous  notation  for  (2.7). 

We  say  that  the  solution  of  (3.1)  has  a  continuous  dependence  on  perturbations  if  x(t\y  + 
6y )  converges  to  x(t\y)  on  each  compact  f-set  as  6y  — >  0.  The  function  /(•)  is  said  to  satisfy 
the  Kamke  (or  simply  the  /7-condition)  if  for  any  x,y  and  i,  satisfying  x  <  y  and  ay  =  yi,  we 
have  fi(x)  <  fi(y).  In  our  case,  f{6)  =  h(9)  —  9,  and  the  condition  holds.  The  //-condition 
implies  the  following  monotonicity  result.  Its  proof  in  [9,  Proposition  1.1]  assumes  continuous 
differentiability  of  /(•).  But,  it  used  only  the  //-condition,  uniqueness  of  solutions,  and  the 
continuous  dependence  of  the  path  on  perturbations,  all  of  which  hold  if  /(•)  is  only  Lipschitz 
continuous. 
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Theorem  4.1.  [9,  Proposition  1.1]  Let  /(•)  be  Lipschitz  continuous  and  assume  the  K- 
condition.  If  x(0)  <  ?/(0)  (resp.,  <,-C,  then  a;(t|a:(0))  <  x(t||/(0)))  (resp.,  <,-C). 

Proof  of  Theorem  2.2.  By  Theorem  2.1,  we  need  only  prove  the  assertions  concerning  the 
limit  points  of  the  paths  of  the  mean  ODE  (2.7).  Because  of  the  boundedness  of  h(-),  the 
path  0(t|0(O))  will  be  bounded,  uniformly  in  the  initial  condition  in  any  compact  set,  and  all 
paths  tend  to  some  compact  set  as  t  — >  oo.  Also,  without  loss  of  generality  (say,  by  shifting 
the  time  origin),  we  can  suppose  where  needed  that  there  is  6  >  0  such  that  0j(i|0(O))  >  6 
for  all  t.  Since  the  path  is  initially  monotone  increasing  (in  each  coordinate)  when  started 
near  the  origin  (since  h{9)  —  9  3>  0  for  6  near  the  origin),  it  follows  from  the  monotonicity 
property  (Theorem  4.1),  that  it  will  be  monotonic  (nondecreasing)  in  each  coordinate  for  all 
t,  for  any  initial  condition  sufficiently  close  to  the  origin.  Thus,  there  is  a  unique  limit  point 
for  the  path  0(f|0(O))  for  each  0(0)  near  the  origin.  By  the  monotonicity,  0(f|0(O))  <  9  for 
all  0(0)  sufficiently  close  to  the  origin  and  all  t. 

Let  0  and  0  be  two  such  limit  points  for  paths  corresponding  to  initial  conditions  0(0) 
and  0(0),  resp.,  arbitrarily  close  to  the  origin.  The  next  step  is  to  show  that  0  =  0.  All  com¬ 
ponents  of  all  paths  that  start  very  close  to  the  origin  are  initially  monotonically  increasing. 
Thus  for  small  enough  0,  we  must  have  0(fo|0(O))  0(0)  for  some  to  >  0.  Then,  by  mono¬ 

tonicity,  0(t  +  to|0(O))  =  0(t|0(to|0(O)))  >  0(f|0(O))  for  all  f  >  0.  Thus,  0  >  0.  An  analogous 
argument  yields  that  0  >  0.  We  can  conclude  that  there  is  a  unique  limit  point,  say  0,  for 
all  paths  starting  sufficiently  close  to  the  origin.  Any  limit  point  must  be  an  equilibrium  for 
(2.7)  in  that  h(9)  =9. 

Now,  consider  the  path  starting  at  an  arbitrary  initial  condition  0  <  0.  After  some  small 
time  to  >  0,  all  components  of  the  path  will  be  positive.  Hence  there  is  0(0)  3>  0,  and 
arbitrarily  close  to  the  origin,  such  that  0(tqJ0)  >  0(0).  Then,  the  monotonicity  argument 
of  the  above  paragraph  yields  that  9{t  +  t0\9 )  >  0(t|0(O))  for  all  t.  Hence  any  limit  point 
of  0(f|0)  must  be  no  smaller  than  0,  the  limit  point  of  0(t 1 0(0) ) .  But,  by  the  monotonicity 
again,  0(7 1 0)  <  0(t|0)  =  0  for  all  t  (the  equality  holds  since  0  is  an  equilibrium  point).  We 
can  conclude  that  0(7 1 0)  — >  0  as  t  — >  oo. 

Define  the  set  Q{9)  =  {x  :  x  >  0}.  Now,  consider  an  arbitrary  initial  condition  0(0). 
The  monotonicity  argument  can  be  used  again  to  show  that  all  limit  points  of  the  path  are 
in  Q(9).  It  only  remains  to  show  that  any  path  starting  in  Q(9)  must  ultimately  go  to  0 
also.  So  far,  we  have  used  only  the  monotonicity  property  and  not  any  other  aspect  of  the 
original  stochastic  approximation  process  that  led  to  (2.7).  The  rest  of  the  details  involve  the 
properties  of  the  argmax  rule  and  essentially  standard  stochastic  approximation  arguments. 
Suppose  that  there  is  a  point  0  e  Q(9),  9  ^  0,  such  that 

u(0)  =  £  (hi(6)  -  9i)/(di  +  6i)  >  0.  (3.2) 
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Since  9  >  9  and  9^9,  (3.2)  implies  that 

J2(hi(0)-9i)/(di  +  9i)>  0.  (3.3) 

i 

Consider  the  algorithm  (1.4)  started  at  9 ,  but  with  the  slot  allocation  rule 

argmax  {rvra+i/(d,:  +  0,;)} 

i<N  ^  > 


used  at  time  n.  Let  /|n+1  denote  the  indicator  function  of  the  event  that  user  i  is  chosen  at 
time  n.  Modulo  a  second  order  error  of  order  0(e)t,  the  expansion  (1.9)  and  the  maximizing 
property  of  J*+1  (see  (1.6))  yield 


u(m)  -  u{9 )  = 


m- 1 

E 

1=0 


Ti,l+1  / 


i,l+ 1 


7i,l 


(1, 


+  9t,i 


>  e 


—  ^ 


m- 1 

E 

1=0 


ri,l+lI'l 


i,l+ 1 


7i,l 


(1, 


+  ^ 'll 


(3.4) 


where  [i/e]  denotes  the  integer  part  of  t/e,  and  9f  (with  interpolation  9e( •))  is  the  solution 
to  (1.4)  under  (1.6). 

The  stochastic  approximation  arguments  that  led  to  Theorem  2.1,  together  with  (3.4), 
imply  that  as  e  — >  0  the  limit  O(-)  satisfies 


U(9(t))  ~  U{9 ) 


hi{9{s))-9i{s)dc> 
di  +  9i(s) 


rt  hj{6)  -  9j(s ) 
Jo  i  di  +  9i(s) 


This,  together  with  the  inequality  in  (3.3)  implies  that 


U(9(t))\t= o 


^  hi(9)  ~9t>  hj{9 )  -  9, 

i  di  +  9i  di  +  9j 


>  0. 


But  the  first  sum  is  zero  since  h{9)  =  9.  Thus,  we  have  a  contradiction  to  (3.2)  and  can 
conclude  that  U(9)  <0  for  all  9  e  Q(9)  —  9.  This  implies  that  U{9{-\9))  is  strictly  decreasing 
when  the  path  is  in  Q{9)  —  9 ,  which  implies  that  any  path  starting  at  some  0(0)  G  Q{9 )  must 
end  up  at  9.  Thus,  9  is  the  unique  limit  point  of  (2.7),  irrespective  of  the  initial  condition. 
Hence  it  is  asymptotically  stable.  ■ 

A  two-user  example.  Consider  two  independent  users  with  received  signal  power  deter¬ 
mined  by  stationary  Rayleigh  fading  (the  Jakes  model)  and  with  constant  external  noise. 
Suppose  further  that  their  rate  declarations  are  proportional  to  the  absolute  SNR,  with  mean 
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rates  1/Pi,i  =  1,2  respectively.  Then  the  function  h  in  (2.7)  can  be  explicitly  evaluated  and 
we  get 

Q  =  J; _ Pl(dl  +  9i)2 _ ^ 

Pi  (Pl(di  +  9\)  +  /?2 (<^2  +  $2))  /o 

g  1  /32(d2  +  g2)2  (d-5j 

P2  (Pi{di  +  6*1)  +  /32(d2  +  6>2))2 

For  dt  —  0,  i  —  1,2,  it  is  readily  shown  that  9i  =  3/4  •  1  / fd%  is  a  limit  point.  More  generally 
9i  =  G(N)/N  ■  1/Pi  where  G(N )  =  J^jLi  1  / j-  Note  that  this  represents  a  scheduling  gain, 
since  with  simple  TDMA  we  would  have  9j  =  1  /\N  P/\.  This  fact  was  also  noted  in  [6],  under 
the  special  conditions  used  there.  The  fact  that  the  limit  throughput  is  proportional  to  the 
mean  rate  is  a  consequence  of  the  facts  that  the  stationary  distribution  for  Rayleigh  fading  is 
exponential,  the  channels  mutually  independent,  and  the  scheduling  rule  is  (1.6),  the  last  fact 
being  a  consequence  of  the  logarithmic  utility  function.  In  [4],  under  the  same  conditions  it 
is  shown  that  all  users  get  the  same  fraction  of  slots,  asymptotically.  The  argument  supposes 
that  there  is  an  equilibrium  point,  a  fact  proved  here. 

Proof  of  Theorem  2.3:  Optimality  properties  of  the  rule  (1.6).  We  work  with  (1.4) 
for  notational  convenience.  Suppose  that  there  is  an  initial  condition  and  a  slot  assignment 
policy  that  depends  only  on  the  available  data  and  which  attains  some  limit  point  9^9 
such  that  U{9 )  >  1/(0);  i.e.,  there  is  an  admissible  assignment  sequence  {/;,«;  f  <  N,  n  <  00} 
under  which  the  weak  sense  limit  is  9  starting  with  the  some  given  initial  condition. 

Now,  consider  the  assignment  algorithm  which  starts  at  the  point  9  at  time  n,  but  uses 
the  “tilde  ”  strategy.  This  yields 


m 

9\ (ne  + 1)  =  (1  -  e)[t/e]9i  +  e  ^(1  -  e)[t/e]^lrijn+ilijn+i.  (3.6) 

1=1 

Under  (A2.0)  and  either  (A2.1a)  or  (A2.2a),  as  e  — »  0  and  n  — >  00,  this  converges  weakly  to 
the  process  with  values  e~t9i  +  \l  —  e~t}9i.  This,  in  turn,  implies  that  the  limit  (as  e  — >  0)  path 
goes  in  a  straight  line  from  9  to  9  ^  9.  Since  U{9 )  >  U(9),  the  rate  of  increase  of  the  utility 
!/(•)  along  that  line  is  strictly  positive  (this  is  where  the  strict  concavity  of  !/(•)  is  used). 
An  argument  that  exploits  the  maximizing  property  of  the  rule  (1.6)  and  the  equilibrium  of 
9  for  (2.7),  such  as  that  used  in  the  proof  of  Theorem  2.2,  shows  that  it  is  impossible  to  have 
such  a  path  in  a  neighborhood  of  9.  ■ 


14 


4  Numerical  Results 


The  graphs  are  from  simulations  based  on  Rayleigh  fading,  and  the  relation  between  the 
current  rates  and  signal  to  noise  ratios  is  taken  from  Table  1,  which  comes  from  [3].  Our 
first  results  depict  the  advantages  to  be  gained  by  taking  advantage  of  the  current  values  of 
the  time  varying  rates.  In  Figure  1,  one  set  of  curves  corresponds  to  the  transient  behavior  for 
three  mobiles  using  table  1  and  mean  SNRs,  -12dB,-2dB,-8dB,  respectively,  using  algorithm 
(1.4).  There  are  two  sets  of  curves:  those  with  solid  lines  and  (the  higher  ones)  those  with 
dotted  lines.  The  solid  lines  depict  the  throughputs  if  the  SNRs  (and  hence  the  rates)  are 
assumed  to  be  constant  at  the  average  values.  We  take  e  =  0.0001.  The  value  of  e  is 
determined  by  a  balance  between  what  is  considered  a  reasonable  measure  of  discounted 
throughput  and  the  desire  to  track  changing  conditions.  If  there  are  1000  slots  per  second, 
then  (roughly  speaking)  e  =  0.0001  corresponds  to  a  measure  over  about  10  seconds.  Initially 
slots  are  offered  only  to  mobile  2,  with  the  throughputs  for  the  other  two  mobile  exponentially 
decaying.  Also  there  are  two  “switching  times”.  At  first,  slots  are  equally  divided  between 
mobiles  2  and  3  (0, 1/2, 1/2),  then  the  slots  are  divided  as  (1/3, 1/3, 1/3).  (This  behavior  is 
generic  for  constant  rates.) 

The  second  set  of  curves  (dotted  lines/filled  symbol)  are  obtained  for  Rayleigh  fading 
with  fading  rate  6  Hz  and  the  same  mean  SNRs.  The  true  current  rates  are  used.  The 
significant  gains  in  the  throughput  for  all  mobiles  are  evident.  Since  the  dependence  on 
rates  is  roughly  linear  on  absolute  SNR,  it  is  expected  the  slots  will  be  approximately  evenly 
divided  in  equilibrium  as  the  users  all  have  exponential  SNR  distributions. 

Example.  Consider  a  case  with  two  users  with  received  signal  power  determined  by  a 
stationary  Rayleigh  fading  process  and  with  constant  and  white  external  noise.  Suppose 
further  that  their  rate  declarations  are  proportional  to  the  SNR,  with  mean  rates  1  / /?i ,  1  / /?2 
respectively.  Using  algorithm  (1.4),  the  ODE  is  (3.5).  For  two  such  users  and  initial  through¬ 
put  250.0,  Figure  2  shows  a  sample  path  for  the  values  of  9  based  on  the  proportional  fair 
throughput  algorithm  with  e  =  0.0001,  as  well  as  a  numerical  solution  to  the  corresponding 
ODE.  The  sample  path  rates  were  given  via  a  Rayleigh  fading  simulator  with  l/j3\  —  572 
bits/slot  and  1  / /32  =  128  bits/slot.  The  fading  rates  were  taken  as  60  Hz.  With  smaller 
values  of  e  and/or  lower  fading  rates  the  sample  paths  fluctuate  somewhat  more  about  the 
solution  to  the  ODE.  In  equilibrium  the  throughputs  are,  §  •  \  ■  572  =  429,  |  •  \  ■  128  =  96. 

The  time  constant  for  convergence  is  1/e  intervals  and  the  results  confirm  convergence  in 
a  period  of  this  order.  The  results  also  show  the  theoretical  equilibrium  being  approached. 
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5  Extensions 


5.1  Other  Utilities  and  Allocation  Rules 

As  noted  in  Sections  3  and  4.  the  algorithm  (1.6)  is  based  on  the  utility  function  U{9)  = 
£*  l°g(di  +  6j).  Other  strictly  concave  utility  functions  can  be  used  as  well  and  there  are 
several  that  seem  advantageous.  One  class  is  described  next.  Consider  the  utility 


U(0)  =  J2ci/(9i  + di)1,  0  <  7  <  1,  Cj  >  0,  di  >  0.  (5.1) 

i 


Then 


Thus  the  chosen  user  is 


vm  =  iE 

i 


(-i  0 

{di  +  di)1-7 


arg  max 

i<N 


QU,n+l 

{6in  +  dj)1_7 


(5.2) 

(5.3) 


The  ODE  is  /  =  Ert I{Ciri/Cj r . >[(0l+dl) /e:j +d:j )] i -y j y *}  -  #*■  The  rule  (5.3)  is  not  as  sensitive  to 
large  values  of  6t  as  is  (1.6).  The  analogs  of  Theorems  2. 1-2.3  hold.  Thus,  there  is  a  wide 
choice  of  useful  and  convergent  algorithms  which  allow  a  variety  of  tradeoffs  between  the 
current  rates  and  throughputs  in  making  the  assignment. 


5.2  Extension  to  Multiple  Channels  and/or  Antennas 

Up  to  this  point,  there  was  only  a  single  resource  (say,  a  transmitter)  to  be  assigned.  There 
are  similar  algorithms  and  results  when  there  are  multiple  resources  to  be  assigned.  To  illus¬ 
trate  some  of  the  possibilities,  consider  the  following  form,  where  there  are  two  transmitters 
to  be  assigned,  with  possibly  different  locations  and  frequencies,  but  at  the  same  base  sta¬ 
tion.  The  associated  channels  will  usually  have  different  characteristics.  For  simplicity  in 
exposition,  we  suppose  that  each  user  has  an  infinite  backlog  of  data  to  be  sent,  and  base  the 
assignment  rule  on  the  utility  (1.8)  and  the  discounted  throughput  as  in  (1.4).  In  general, 
any  number  of  antennas  can  be  used. 

One  can  allow  many  alternatives  in  the  way  that  the  transmitters  are  assigned.  The 
examples  are  only  intended  to  be  illustrative  of  the  possibilities  that  can  be  handled  by  the 
approach.  In  all  cases  to  be  discussed,  it  is  assumed  that  the  receiver  at  the  mobiles  are 
equipped  to  handle  the  method.  The  simplest  method  of  assignment  is  to  assign  each  of  the 
two  via  an  analog  of  (1.6)  just  as  the  single  transmitter  was  assigned  in  Section  2.  Then 
both  might  be  assigned  to  one  user,  or  only  one  might  be  assigned  to  each.  The  assignment 
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algorithm  is  just  (1.6),  applied  to  each  transmitter  separately.  Equivalently,  one  assigns 
so  as  to  maximize  the  first  order  term  in  U[9£n+l)  —  U(9€n),  where  U(-)  is  defined  by  (1.8). 
The  assumptions  are  just  those  of  Theorems  2. 1-2.3,  applied  to  the  channels  from  each 
transmitter  separately.  In  particular,  there  is  a  unique  globally  asymptotically  stable  limit 
point  6,  and  the  argmax  assignment  algorithm  maximizes  the  utility.  The  basic  properties 
that  were  used  in  the  proofs  still  hold.  In  particular,  the  mean  ODE  still  satisfies  the  Kamke 
condition,  and  the  argmax  rule  maximizes  the  increment  in  the  utility  to  first  order.  Let  r^jn 
denote  the  canonical  rates  for  user  i  in  the  channel  from  transmitter  j  at  scheduling  interval 
n.  For  the  two  user  case,  the  iteration  is 


#1,77+1  =  (1  -  e)  #1,77  +  er ll,n+l-f{ni,n+i/r-2i,„+i>(<ii+^>n)/(d2+01in)} 

+er  12,n+lI{ri2,n+i/r22,n+i>(di+0ln)/(d2+e^J}, 

with  the  analogous  formula  for  the  other  user. 

There  was  no  coordination  between  the  assignments  of  the  two  transmitters  in  the  method 
just  discussed.  Next  consider  an  alternative  that  allows  for  more  efficient  use  of  the  resource. 
We  still  allow  the  above  choices,  where  each  transmitter  is  assigned  independently.  But  now 
we  allow,  in  addition,  the  possibility  of  the  two  antennas  being  used  in  a  coordinated  way 
for  the  same  user,  with  (for  example)  space-time  coding.  This  simply  adds  another  possible 
rate  to  be  considered  when  using  the  argmax  rule  when  making  the  assignment.  Space-time 
coding  is  selected  simply  because  it  is  one  way  of  using  both  channels  for  the  same  user.  The 
choice  is  still  made  for  each  scheduling  interval,  and  might  differ  from  interval  to  interval. 
The  full  assignment  algorithm  increases  the  channel  capacity  over  what  space-time  coding 
used  by  itself  could  achieve.  Let  rfn  denote  the  rate  using  space-time  coding  when  both 
channels  are  assigned  to  user  i  and  let  I denote  the  indicator  of  this  event.  It  is  usually  the 
case  that  rcin  >  rnjn  +  r^n,  and  we  make  this  assumption.  For  simplicity  in  the  notation, 
the  discussion  is  restricted  to  the  case  of  two  users. 

Let  Ifj  denote  the  indicator  function  of  the  event  that  user  i  is  assigned  to  channel  j  in 
interval  n  but  space-time  coding  is  not  used.  Clearly,  we  need  Z)iK^-  n  + A>]  =  1  f°r  j  =  1,  2, 
Then,  to  first  order  in  e,  U (#()+1)  —  U(9^)  equals  e  times: 


D  1,77+1-*  11,71+1  I  r12,n+l 


d\  +  9f , 


+ 


rc  T£,c 
*  1,71+1-*  1,71+1 

d\  +  9\r 


d\  +  9 1 , 


+ 


d>2  +  9'. 


+ 


rc  Je,c 

1  2,77+1  -*2,77+1  _ 


2,71 

^1,77 


d-2  +  6\ 


2,n 


d\  +  9- 


'^22,77+1-^22,77+1 

d’2  +  #2,77 

^2,77 

d‘2  +  #2,77 


This  yields  a  slightly  more  complicated  form  of  the  arg  max  rule  of  (1.6).  One  looks  for  the 


IT 


maximum  of 


f  ll,n+l 


+ 


f  22,n+l  f  12,n+l 


+ 


r  21,n+l  rl,n+l  r2,n+l 


@l,n  +  d\  @2,n  +  ^2  #l,n  +  ^1  $2,n  +  ^2  #l,n  +  <A  $2,n  +  ^2 


Define  i?n  =  (riijn,ri2in,'f'2i,n,^22,n,i''inir2n)-  Suppose  that  Rn  is  bounded,  stationary,  and 
has  a  bounded  density.  Then,  under  the  natural  analogs  of  the  other  parts  of  (A2.0)  and 
(A2.1a),  the  analysis  of  the  resulting  algorithm  is  similar  to  what  was  done  for  (1.2)  and 
(2.4),  and  the  analogs  of  Theorems  2. 1-2. 3  hold.  Thus,  even  for  this  more  complicated  case, 
there  is  a  unique  limit  point  and  the  algorithm  is  a  utility  maximizer. 


6  A  General  data  Arrival  Model 


The  formulation  in  Section  1  supposed  that  each  user  always  has  an  infinite  amount  of  data 
to  be  sent.  This  is,  in  fact,  a  shortcoming  of  the  literature  to  date.  Consider  an  alternative 
model,  where  there  are  still  N  users,  but  data  arrives  at  random  and  is  queued,  awaiting 
transmission,  and  an  argmax  discipline  such  as  (1.6)  is  used.  We  will  confine  our  attention 
to  the  throughput  as  measured  by  (1.3),  although  (1.1)  could  be  used  as  well.  Let  Q\n 
denote  the  content  of  the  queue  for  user  i  at  time  n.  Define  Qen  =  {Q\n)i  <  N}  and 
Wfin  =  min{ryn,  Q\ n_i}.  The  decisions  for  the  (n  +  l)st  interval  are  still  made  at  time  n, 
when  Rn+i  is  known.  We  will  suppose  that  each  queue  i  has  a  finite  buffer  of  size  £>* ,  and 
inputs  to  a  full  buffer  are  rejected  and  disappear  from  the  system. 

If  Q\n  >  ritn+i  and  queue  i  is  selected,  then  an  amount  rijU+i  is  transmitted  and  the 
queue  decreases  by  that  amount.  But,  if  0  <  Qein  <  rin+\:i  then  one  needs  to  modify  the 
algorithm  to  reflect  the  fact  that  if  queue  i  is  selected,  then  the  entire  slot  won’t  be  filled. 
There  are  many  ways  of  dealing  with  this  problem.  We  will  choose  the  practical  approach 
of  assuming  that  the  decision  is  made  on  the  basis  of  the  current  rates  and  the  current  value 
of  the  throughput,  as  in  (1.6).  More  particularly,  the  decision  at  time  Rl  is 


arg  max 


{ri,i+ 1/ (di  +  9^)  | . 


(6.1) 


The  throughput  is  updated  as  (which  defines  Fne) 


Oi,n+ 1  —  Oi,n  +  6 


ri,n+llin+l 


l{Qi,n>  0} 


—  9j„  T 


(6.2) 


The  motivation  for  (6.2)  is  that  the  scheduler  will  know  only  the  rates  and  whether  or  not  a 
queue  is  empty,  but  not  the  content  of  the  queue.  The  scheme  can  be  adjusted  in  many  ways, 
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taking  into  account  whatever  information  is  available.  For  example,  after  a  queue  reaches 
zero,  there  might  be  a  latency  period  before  it  is  polled  again.  The  queues  evolve  as 


Qi,n+ 1  Qi,n  P  ^i,n+l 


z,n+l  i,n+l 


(6.3) 


where  &4vn+i  denotes  the  number  of  arrivals  to  queue  i  in  time  slot  n  +  1 ,  and  Ufn+l  denotes 
the  amount  of  data  that  was  rejected  due  to  a  full  buffer. 

The  model  for  the  evolution  of  the  queue  and  the  will  now  be  specified.  We  have 

in  mind  that  there  are  essentially  continuous  arrivals  to  the  queues,  but  at  randomly  time 
varying  rates.  The  pair  (Q^,rn+ 1),  together  with  8An+\  and  9m  determines  QiiU+ 1-  The  pair 
(Qn,rn+ 1),  together  with  9fn ,  determines  9en+v  The  process  {Rn,n  <  ex)}  does  not  depend 
on  the  evolution  of  the  throughputs  9n  in  that 


P  {Rn+i  G  "I Ri,9[,l  <  n}  —  P  G  -\Ri,l  <  n}  . 


But 

P  {Qn+ 1  G  '|  Qli@l,l  E  7 ^  P  {Qn+1  G  '|  Ql,l  <  n}  , 

since  the  values  of  the  9en  still  influence  the  assignments  when  some  queues  are  empty  or 
nearly  empty.  The  contents  of  the  queues,  on  the  other  hand,  helps  determine  the  evolution 
of  the  throughputs.  The  Qfn  (together  with  the  Rn )  play  the  role  of  “noise”  and  the  9en  are  the 
“states”  of  the  system  and  we  have  what  is  called  “state-dependent”  noise  [8,  Chapters  6,  8] 
(at  least  the  Q- component  is  state-dependent).  The  assumptions  are  stated  in  a  somewhat 
abstract  way  since  we  wish  to  cover  as  many  cases  as  possible  within  a  single  framework. 

The  main  issue  in  the  convergence  proof  concerns  the  fact  that  the  evolution  of  the  noise 
is  determined  by  that  of  the  throughputs,  unlike  the  situation  in  Section  4.  This  complicates 
the  averaging  and  use  of  conditions  such  as  (2.2).  But  the  fact  that  9en  varies  slowly  for  small 
e  will  help.  Redefine  the  “memory”  random  variables  to  be  =  {Ri,  l  —  n,  n  —  1, . . .}. 
Let  Efn  denote  the  expectation,  conditioned  on  the  new  For  each  n  and  9,  define  the 
fixed-9  process  {Q(9),l  >  n}  as  follows:  Suppose  that  after  time  n,  we  use  the  fixed  value 
9  in  (6.1)  instead  of  the  true  current  throughput.  For  l  >  n,  we  define  it  to  be  Q€i(9),  with 
“initial”  condition  Qen(9)  =  Qew  since  the  change  is  for  times  /  >  n  only.  Define  the  functions 


Cn)  -  Enri,n+1  /{r<,n+l/(di+ep>rJ- n+l/(d3+^), 


z>o  }hQl 


>0}- 


(6.4) 


We  will  need  the  following  conditions.  They  are  quite  weak,  and  their  reasonableness  will 
be  seen  by  the  example  below. 

(A8.1)  There  are  Lipschitz  continuous  functions  <  N,  such  that  as  m  — >  oo 


— E 
m 


n+m—  1 


l=n 


E  En  hi>l(9,^(9))-hl(9) 


(6.6) 
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uniformly  in  n,  e,  and  in  9  in  any  compact  set.  There  is  ho  >  0  such  that  h,  ( • )  >  /i0  for  small 
Oi. 

(A8.2)  For  each  integer  m, 


m 


n+ra— 1 

E  KKiKMOn)) 

l=n 


o, 


uniformly  in  n,  as  e  — >  0. 

(A8.3)  For  each  integer  m  and  as  \9  —  9\  — >  0, 


— E 
m 


n+m— 1 


l=n 


0, 


(6.5) 


(6.7) 


uniformly  in  n  and  e,  where  9  and  9  are  in  any  compact  set. 

Example  and  discussion  of  the  conditions.  The  conditions  are  intuitively  reasonable 
and  the  Jakes  model  of  Rayleigh  fading  is  again  covered.  Conditions  (A8.2)  and  (A8.3) 
are  basically  conditions  on  the  sensitivity  of  the  “conditional  expectation  of  the  amount 
transmitted”  for  user  i  to  very  small  changes  in  the  throughput  that  is  used  to  make  the 
decisions.  Let  us  examine  (A8.2)  with  n  <  l  <  n  +  m.  The  difference  between  the  terms 
Efnhi,i(9)n,  (+))  and  <+  is  that  the  second  is  the  conditional  expectation  of  the  true 

amount  transmitted  for  user  i  at  /,  given  the  data  to  n,  and  the  first  term  is  the  conditional 
expectation  of  the  amount  that  would  have  been  transmitted  if  the  rule  (6.4)  were  used  with 
9  =  9en.  However,  over  the  time  [n,  n  +  me),  the  change  in  9\  is  bounded  by  eme,  which  goes 
to  zero  as  e  — >  0.  Thus,  the  value  of  the  state  at  the  times  of  the  summands  in  (6.5)  is 
arbitrarily  close  to  9en ,  uniformly  in  n.  Condition  (A8.3)  is  similar.  It  says  that  if  we  make 
the  decisions  on  [n,  n  +  me)  always  using  either  9  or  9 ,  that  are  very  close  to  one  another, 
then  the  conditional  mean  amounts  transmitted  on  that  interval  are  also  very  close. 

Let  us  illustrate  the  above  comment  via  a  simple  example.  Let  {/?„}  and  {&4n}  be  mutu¬ 
ally  independent,  with  the  members  of  the  latter  sequence  being  mutually  independent  and 
identically  distributed.  Suppose  that  the  component  sequences  {ryn,  n  <  oo}  are  mutually 
independent  in  i.  Let  there  be  0  <  <  1,  such  that  ryn+i  =  cqryn  +  +n,  where  for  each 

i,  the  6ritn  are  mutually  independent,  identically  distributed,  bounded,  and  have  a  bounded 
and  continuous  density.  Then,  due  to  the  Markov  property,  we  can  use  ££  =  ( Qn,Rn ).  In 
(6.4),  we  have  ritU+ 1  =  cqrjin  +  where  the  8r^n  are  bounded,  mutually  independent,  and 
have  a  bounded  density.  This  independence  and  density  properties  imply  that  small  changes 
in  the  value  of  9  used  in  (6.4)  changes  the  value  of  (6.4)  only  slightly,  provided  that  the 
set  of  empty  queues  does  not  change.  Furthermore,  since  the  Rn  are  bounded,  over  any  m 
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iterates  the  value  of  9en  changes  by  at  most  Km  for  some  constant  K.  By  using  the  above 
facts  and  working  forward  from  iterate  to  iterate,  it  can  be  seen  that  the  probability  that 
the  decision  at  any  /  e  [n,  n  +  m]  will  be  different  (implying  the  possibility  that  the  set  of 
empty  queues  might  change)  for  the  two  assignment  methods  goes  to  zero  uniformly  in  n  as 
e  — >  0.  Thus  (A8.2)  holds.  Similarly,  the  probability  that  any  assignment  in  [n,n  +  m\  based 
on  the  argmax  rule  using  6  will  differ  from  that  based  on  9  also  goes  to  zero  as  \9  —  9\  — »  0. 
Thus  (A8.3)  holds. 

(A8.1)  is  a  weak  ergodic  condition.  Let  us  make  the  decisions  using  the  value  9  for  all 
n  >  0,  and  not  the  true  throughput.  This  yields  the  fixed-0  Markov  process  which  we  call 
£n(9)  =  ( Rn,Qn(9 ),  for  n  >  0,  with  some  arbitrary  initial  condition  £0.  The  process  £n(9) 
has  a  unique  invariant  measure.  Then,  owing  to  the  fact  that  the  transition  probability 
of  the  Markov  process  does  not  depend  on  time,  (A8.1)  says  nothing  more  than  that  the 
conditional  expectation  (given  £o)  °f  the  throughput  on  [0,  n]  corresponding  to  the  continual 
use  of  the  value  9,  converges  to  the  average  value  as  n  — >  oo.  The  convergence  is  uniform  in 
the  initial  data  and  in  the  value  of  9  in  any  compact  set. 

Although  the  Rayleigh  fading  process  is  not  Markovian,  it  has  “mixing  and  density” 
properties  that  are  similar  to  our  Markov  example. 

Theorem  8.1.  Assume  (A8.1)-(A8.3).  Then  the  conclusions  of  Theorems  2. 1-2.3  hold. 

Comment  on  the  proof.  Conditions  (A8.2)  and  (A8.3),  then  (A8.1),  (A8.2),  and  (A8.3), 
are  close  to  conditions  (A4.6),  (A4.16'),  and  (A4.17'),  resp.,  of  [8,  Theorem  4.4,  Chapter  8], 
and  can  replace  them  in  the  proof  of  the  cited  theorem.  Theorem  4.4  in  [8,  Chapter  8]  is  an 
analog  for  the  state  dependent  noise  case  of  the  result  cited  in  Theorem  2.1,  and  assures  the 
conclusions  of  Theorem  2.1.  The  conclusions  of  Theorem  2.2  hold  since  they  depend  only  on 
certain  properties  of  the  mean  ODE,  which  hold  in  the  present  case.  Theorem  2.3  depends 
on  the  stochastic  approximation  arguments  which  led  to  Theorem  2.1,  the  uniqueness  of  the 
limit  point,  and  the  strict  concavity  of  the  utility  function,  all  of  which  hold  in  the  present 
case. 
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Figure  1:  Time  dependent  behavior  of  Proportional  Fair 


Figure  2:  Sample  path  for  8  and  the  Solution  to  the  ODE 


<  SNR 

-12.5 

-9.5 

-8.5 

-6.5 

-5.7 

-4.0 

Rate 

0.0 

38.4 

76.8 

102.6 

153.6 

204.8 

<  SNR 

-1.0 

1.3 

3.0 

7.2 

9.5 

- 

Rate 

307.2 

614.4 

921.6 

1228.8 

1843.2 

2457.6 

Table  1:  Rate  vs.  SNR  for  1%  packet  loss  (taken  from  [3] 
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